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Abstract 

The purpose of the study was to determine the effect of problem-based learning ( PBL ) on critical 
thinking ability and content knowledge among selected secondary agriculture students in 
Missouri. The study employed a quasi-experimental, non-equivalent comparison group design. 
The treatment consisted of two instructional strategies: problem-based learning and supervised 
study. The target population was identified as secondary agriculture students in Missouri. 
Twelve secondary agriculture teachers were selected based on criteria established by the 
researcher. Intact classrooms were randomly assigned to a level of the treatment. The resulting 
sample (n = 140) consisted of 77 students in the problem-based learning treatment group and 63 
students in the supervised study treatment group. Analysis of covariance indicated a treatment 
effect on critical thinking ability and content knowledge. 


Introduction and Theoretical Framework 

“Whether our focus is on classical 
education, the new math, or basics, the 
ultimate goal of education has been to teach 
children to think critically and 
independently” (Sternberg & Baron, 1985, 
p. 40). The origins of critical thinking can be 
traced to the early philosophies of Plato and 
Aristotle. The importance of critical thinking 
was evident in the beginning of the modem 
era of education in the writings of Dewey 
(1909/1997), who described the ability to 
think critically as a way to find meaning in 
the world in which we live. 

Refonn initiatives in education have 
further solidified the concept of critical 
thinking as a requisite goal of education. A 
Nation at Risk (National Commission on 
Excellence in Education, 1983) sounded an 
alarm at our faltering attempts to foster 
critical thinking, higher-order thinking and 
problem solving in our nation’s schools. 
The Secretary’s Commission on Achieving 
Necessary Skills (1991) ranked 
competencies in critical thinking, decision 
making, problem solving, and reasoning as 
imperative for high perfonning workplaces. 
An emphasis is now being placed on the 


ability to understand and use information, 
not just merely possess it (Richardson, 
2003). 

Almost unanimously, educators believe 
the development of critical thinking ability 
should be a primary goal of education 
(Pithers & Soden, 2000). However, the 
actions of educators would suggest 
otherwise. “Three-quarters of a century of 
educational literature suggests the main 
emphasis in schools has been teaching 
students facts, even though teachers and 
curriculum designers have attested to the 
importance of teaching students to think” 
(Cano & Newcomb, 1990, p. 46). While the 
importance of acquisition and recall of basic 
knowledge remains important, the 
development of critical thinking has 
emerged as equally important. Can we find 
a balance with instructional strategies that 
facilitate the acquisition of basic knowledge 
yet develop and nurture critical thinking? 

Driscoll (1994) provided a conceptual 
framework for distinguishing theories of 
instruction from theories of learning. 
Driscoll stated that while the two must be 
compatible, they represent different views of 
learning outcomes. According to the model 
(Figure 1), learning theories attempt to 
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explain the interaction of required learning 
conditions and outcomes of learning. A 
foundational assumption is that learning 
occurs when conditions are ripe. In contrast 
to learning theories, instructional theories 
account for “a deliberate arrangement of 
learning conditions to promote the 
attainment of some intended goal” (Driscoll, 
p. 332). 

Problem-based learning (PBL) is a 
constructivist approach to instruction that 
revolves around a real-world, ill-structured 
problem (Jonassen, 1997). PBL promotes 
both the acquisition of content knowledge 
and the development of thinking skills and 
strategies. Teachers typically take on the 
role of the facilitator and students become 
responsible for information learned. This 
method typically ends with a presentation of 
solutions and an evaluation of the process 
used in solving the problem. PBL was 
pioneered as a method of instruction in 
medical programs and was designed with 
several important goals. It was designed to 
help students construct an extensive and 
flexible knowledge base; develop effective 
problem-solving strategies; develop self- 
directed, lifelong learning skills; become 
effective collaborators; and become 
intrinsically motivated to learn (Barrows & 
Kelson, 1995). In recent years, PBL gained 
in popularity at the collegiate and secondary 
levels as well. 

Studies have explored the outcomes 
related to PBL at virtually all levels of 
education. There is agreement on the 
contribution of PBL to factors such as 
knowledge retention, student satisfaction, 
motivation, and critical thinking. There is 
much less agreement on the role of PBL in 
knowledge acquisition. Vernon and Blake 
(1993) concluded that PBL students are at a 
disadvantage when compared to traditional 
students on content knowledge. Albanese 
and Mitchell (1993) were much less 
confident, asserting that PBL students are at 
a disadvantage sometimes, but not always. 
Albanese and Mitchell attributed the 
disparity to the variation in which PBL was 
implemented. Others (Alleyne, et ah, 2002; 
Dods, 1997; Leiux, 1996) found no 
difference in the content knowledge of 
students exposed to PBL compared to 
traditional instructional strategies. 


Evidence suggests PBL can help 
promote critical thinking skills. Studies 
investigating problem-solving, a component 
of critical thinking, have found that students 
exposed to PBL consistently display growth 
in problem-solving skills (Ball & Knobloch, 
2004; Hmelo, 1998). Additionally, students 
in PBL programs showed an increase in 
transfer and application of knowledge 
(Norman & Schmidt, 1992) and in analysis 
and application required in clinical trials 
(Albanese & Mitchell, 1993), each 
considered essential to problem-solving. 
PBL has been found to be effective in 
promoting higher-order thinking (Albanese 
& Mitchell; Cockrell, Caplow, & 
Donaldson, 2000; Dods, 1997; Vernon & 
Blake, 1993). 

The literature additionally suggests a 
general consensus regarding a positive 
impact of PBL on student dispositions. PBL 
has been found to improve student 
motivation and interest (Gordon, Rogers, 
Comfort, Gavala, & McGee, 2001; Herman 
& Knobloch, 2004; Norman & Schmidt, 
1992). In addition to motivation, students 
indicate more satisfaction with PBL than 
with traditional methods of instruction 
(Albanese & Mitchell, 1993; Ball & 
Knobloch, 2004; Cockrell et ah, 2000; 
Gordon et al.; Vernon & Blake, 1993). 

While there are elaborate descriptions of 
using PBL in various settings, there is little 
empirical evidence as to what students are 
learning and how (Hmelo-Silver, 2004). 
Herman and Knobloch (2004) recommended 
future studies investigate the use of 
constructivist PBL approaches to determine 
effects on learning outcomes in agriculture 
classrooms. 

Agricultural education has long valued 
the importance of problem solving (Brown, 
1998). In turn, the problem-solving 
approach has been touted as the most 
effective method for teaching (Osborne & 
Hamzah, 1989). Research validating this 
claim has been inconclusive. While the 
problem-solving approach and problem- 
based learning share common educational 
goals, the two have very different 
philosophical origins. Few studies (Herman 
& Knobloch, 2004) have sought to identify 
the impact of PBL in secondary agriculture 
classrooms. 
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Research on critical thinking has linked 
the development of critical thinking skills to 
instructional design. Lundy et al. (2002) 
concluded critical thinking was a skill that 
can be acquired and developed in all 
students by utilizing critical thinking 
instructional techniques. Instructional design 
has been found to improve critical thinking 
through active learning strategies (Burbach, 
Matkin, & Fritz, 2004) and contextual 
applications (Elliot, Oty, McArthur, & 
Clark, 2001) in college students. In a study 
of 5 th and 6 th grade students, Mabie and 
Baker (1996) concluded that experiential 
learning activities can lead to increased 
critical thinking skills. Less infonnation is 
available on the role PBL plays in the 
critical thinking development of secondary 
students. 

Results from this study will serve to 
answer these concerns, in part, and 
contribute to the knowledge base of teaching 
and learning as a process in education in 
general and specifically in the field of 
agricultural education. Furthermore, 
findings from this research will inform 
practicing teachers of alternative strategies 
that can be used to meet the expectations 
established at the state and national level as 
well as help inform teacher preparation 
programs as to what instructional strategies 
should be taught to pre-service teachers. 

Purpose and Objectives 

The purpose of this study was to 
determine the effect of problem-based 
learning (PBL) on critical thinking ability 
and content knowledge. The following 
research objectives and hypotheses were 
generated to focus and guide the direction of 
the study. 

1. Describe students on gender, grade 
classification, and academic aptitude. 

2. Describe the critical thinking ability 
of students before and after 
instruction in a quail management 
unit. 

3. Describe the content knowledge of 
students before and after instruction 
in a quail management unit. 

4. Compare the effect of instructional 
strategy (problem-based learning 


versus supervised study) with regard 
to secondary agriculture students’ 
critical thinking ability and content 
knowledge. 

Hi: Students taught using the problem- 
based learning instructional strategy 
will demonstrate a greater 
improvement in critical thinking than 
students taught using the supervised 
study instructional strategy. 

H 2 : A significant difference exists in 
content knowledge for students 
taught using the problem-based 
learning instructional strategy and 
students taught using the supervised 
study strategy. 

Methods and Procedures 

A quasi-experimental, non-equivalent 
comparison group design was utilized. The 
design included both pre-test and post-test 
data gathered on the same unit making it a 
dependent samples design. Campbell and 
Stanley (1969) advocated the use of both 
pre-test and post-test when the groups are 
similar, but not so similar that a pre-test is 
unnecessary. Using both a pre-test and a 
control group allows for greater ease in 
examining threats to internal validity 
(Shadish, Cook, & Campbell, 2002). 

The target population for the study 
was identified as secondary agriculture 
students in Missouri. Subjects in the study 
were part of a purposive sample. 
Selection was determined by criteria of the 
instructors. Twelve teachers were selected 
based on characteristics of their teacher 
preparation program. All selected teachers 
had been exposed to a similar pre- 
service departmental philosophy of 
education, completed similar requirements 
for teacher certification, and received similar 
instruction in teaching methodology. 
Teachers were included in the study based 
on their ability to include the Quail 
Management Unit in their Ag Science II or 
Natural Resource/Conservation class. Each 
intact classroom was randomly assigned to 
the supervised study treatment or the 
PBL treatment. Subjects were part of the 
intact classrooms of the selected 
teachers. The resulting sample (n = 140) 
consisted of 77 students in the PBL 
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treatment and 63 students in the supervised 
study treatment. 

Students were taught a unit of instruction 
on quail habitat management developed by 
the Missouri Department of Conservation. 
Six of the teachers taught the unit using the 
supervised study method of instruction as 
described by Newcomb, McCracken, and 
Warmbrod (1993). This strategy represents 
the design of Missouri’s recommended 
curriculum for agriculture classes. The 
remaining six teachers taught the unit using 
the PBL strategy of instruction. Both groups 
received all necessary materials according to 
their respective strategy. Additionally, a 
professional development session was 
conducted to prepare teachers to implement 
the assigned strategy. 

Three data collection instruments were 
used. Critical thinking ability was 
determined by the Watson-Glaser Critical 
Thinking Appraisal" 8 (WGCTA®), Form S. 
Content knowledge was determined by a 
score on a quail management test developed 
by the Missouri Department of Conservation 
for use with the unit of instruction. Finally, 
descriptive information of students (gender, 
grade level, and academic aptitude) was 
reported by the teacher on a report form 
developed by the researcher. 

The WGCTA 8 is a standardized, 
copyrighted assessment tool for assessing 
the success of programs and courses in 
developing critical thinking skills (Watson 
& Glaser, 1994). The instrument includes 
exercises which are purported to be 
examples of problems, statements, 
arguments and interpretations of data which 
are regularly encountered at work as well as 
at school and in other activities. The 
WGCTA® is designed to measure critical 
thinking as a composite of attitudes, 
knowledge, and skills. The instrument is 
available in parallel forms A and B and is 
also available in an abbreviated version, 
form S. Form S was used for this study as it 
is approved for secondary students and can 
be completed in approximately 45 minutes. 

Reliability estimates for Form S of the 
WGCTA were reported as a Cronbach’s 
alpha coefficient of .81 (r = .81) (Watson & 
Glaser, 1994). According to Watson and 
Glaser (1994), “the content validity of the 


WGCTA® in classroom and instructional 
settings may be examined by noting the 
extent to which the WGCTA measures a 
sample of the specified objectives of such 
learning programs” (p. 35). The statewide 
objectives of public education in Missouri 
clearly identify the importance of critical 
thinking skills as evident by the references 
to analysis, problem solving, and decision 
making. The construct validity of the 
WGCTA 8 can be evaluated by noting its 
relationship with other tests. Watson and 
Glaser reported significant relationships 
between the WGCTA and test of general 
intelligence (Otis-Lennon Mental Ability 
Tests, the California Test of Mental 
Maturity, and the Wechsler Adult 
Intelligence Scale Verbal IQ). 

Content knowledge was determined by a 
score for participants on the post-test 
administration of the quail management test 
developed in conjunction with the original 
instructional unit. The test consisted of 50 
selected response items related to four unit 
objectives. The reliability coefficient was 
determined post hoc to be .85 by assessing 
the inter-item consistency according to 
Kuder-Richardson formula 20. The K-R 20 
is applicable to tests whose items are scored 
dichotomously as either right or wrong (Ary, 
Jacobs, & Razavieh, 2002). The content 
items on the quail management test were 
selected by a panel of experts in wildlife 
management and in agricultural education 
who developed the original curriculum. 

Descriptive data for students in each of 
the classes were collected on the score 
report form. Teachers were asked to record 
these data (gender, classification, and 
academic aptitude) in the appropriate field 
of the Score Report Form. That form was 
returned to the researcher at the conclusion 
of the unit. Academic aptitude, operationally 
defined as Missouri Assessment Program 
(MAP) score, was used as a covariate to 
control for differences in aptitude in the 
sample. The MAP is a standardized test 
designed to assess proficiency in 
mathematics, science, communication arts, 
and social studies at a statewide level. 
Students are assessed on each MAP area 
once in elementary, middle, and high school 
grade levels. 
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Findings 

The first objective sought to describe 
students on gender, grade classification, 
and academic aptitude (7 th grade MAP 
index). A total of 140 students participated 
in the study. Approximately two-thirds of 
the students (65%) were male and 
approximately one-third (35%) were female. 
In the problem-based learning (PBL) group 
(n = 77), just over half of the students 
were male (n = 44, 58%), while nearly 
three-fourths of the supervised study 
treatment group (n = 63) was composed of 
males (n = 47, 73%). Sophomores 

represented a majority (n = 79, 56%) of 
the sample followed by seniors (n = 23, 
16.4%), freshmen (21, 15%) and juniors (17, 
12%). When distinguished by group, 
sophomores made up 61.0% (n = 47) of the 
PBL group and 50.8% (n = 32) of the 
supervised study group. The PBL group was 


additionally comprised of 14 seniors 
(18.2%), 10 freshmen (13.0%), and 6 juniors 
(7.8%). The supervised study group was 
represented by 11 freshmen (17.5%), 11 
juniors (17.5%), and 9 seniors (14.3%). 

Academic aptitude was operationally 
defined as the scale score on the 
Missouri Assessment Program (MAP 
Score). The most recent administration 
completed by all students in this study 
was the 7 th grade administration. MAP 
scores were collected in raw score 
form but were reported categorically by 
achievement level for descriptive purposes 
(Table 1). Almost half (44%, n = 46) of the 
students in the sample were in the 
progressing category. Thirty-two percent (n 
= 34) of the sample was categorized as 
nearing proficiency. The remainder of the 
sample consisted of step 1 (n = 13, 12%), 
proficient (n = 11, 1 1%), and advanced (n = 
1 , 1 %). 


Table 1 


Achievement Categories of MAP Scores by Treatment Group (n = 105) 


Achievement Category 

Problem-Based 

Supervised Study 


Total 

f 

% 

/ 

% 

/ 

% 

Step 1 

7 

11.9 

6 

13.0 

13 

12.4 

Progressing 

32 

54.2 

14 

30.4 

46 

43.8 

Nearing Proficiency 

16 

27.1 

18 

39.1 

34 

32.3 

Proficient 

4 

6.8 

7 

15.2 

11 

10.5 

Advanced 

0 

0.0 

1 

2.2 

1 

0.9 

Total 

59 

100.0 

46 

100.0 

105 

100.0 


Note. MAP scores were not available for all students. 


The second objective sought to describe 
the critical thinking ability of students 
before and after instruction in a quail 
management unit. Summary statistics were 
calculated for the pre-test and post-test 
administration of the Watson-Glaser Critical 
Thinking Appraisal* (WGCTA®) (Table 2). 
Sample pre-test scores for both groups 
ranged from 11 to 37 with a mean of 21.3 


(SD 4.5). Post-test scores for the two groups 
ranged from 10 to 33 with a mean of 21.4 
(SD = 4.4). The average pre-test WGCTA 1 ' 
score for the PBL group (n = 77) was 21.1 
(SD = 4.3) and the post-test score was 21.2 
(SD = 4.7). Students in the supervised study 
group (n = 63) achieved an average pre-test 
WGCTA® score of 21.4 (SD = 4.7) and 
post-test score of 21.6 (SD = 4.1). 
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Table 2 

Critical Thinking (ll'CCTA ) Scores by Level o f Treatment 





Pre-Test 



Post-Test 

Treatment 

n 

M 

SD 

Range 

M 

SD 

Range 

Problem-Based Learning 

77 

21.1 

4.30 

11-37 

21.2 

4.68 

10-33 

Supervised Study 

63 

21.4 

4.73 

11-34 

21.6 

4.06 

13-32 

Total 

140 

21.3 

4.49 

11-37 

21.4 

4.39 

10-33 

Objective three sought to describe the 
content knowledge of students before and 
after instruction in a quail management unit. 
Content knowledge was detennined by the 
score on a quail management test. Scores 
were determined by summating the number 
of correct items resulting in a possible score 
of 0 to 50. As displayed in Table 3, sample 
pre-test scores for both groups ranged from 
10 to 35 with a mean of 20.4 ( SD 4.2). Post- 

test scores for the two groups ranged from 8 
to 42 with a mean of 26.4 (SD = 6.9). 
Students in the PBL group (n = 77) correctly 
identified an average of 20.1 items (SD = 
4.3) on the quail management test pre-test 
and 24.2 items (SD = 6.1) on the post-test. 
The average pre-test score for students in the 
Supervised Study group was 20.8 (SD = 4.1) 
and the average post-test score was 29.0 
(SD = 6.9) 

Table 3 

Content Knowledge Scores bv Level of Treatment 








Pre-Test 



Post-Test 

Treatment 

n 

M 

SD 

Range 

M 

SD 

Range 

Problem-Based Learning 

77 

20.1 

4.3 

12-35 

24.2 

6.1 

8-36 

Supervised Study 

63 

20.8 

4.1 

10-30 

29.0 

6.9 

13-42 

Total 

140 

20.4 

4.2 

10-35 

26.4 

6.9 

8-42 


The fourth objective was to compare 
the effect of instructional strategy 
(problem-based learning versus 
supervised study) with regard to 

secondary agriculture students’ critical 
thinking ability and content knowledge. 
The first research hypothesis stated that 
students taught using the problem-based 
learning instructional strategy will 
demonstrate a greater improvement in 
critical thinking than students taught 
using the supervised study instructional 
strategy. 


The null hypothesis was tested using 
ANCOVA to control for critical thinking 
ability prior to instruction (pre-test 
WGCTA ) and for academic aptitude (MAP 
score) (Table 4). The F- value (Fuos = 
10.96) was significant (p = .01) at the alpha 
level of .05, established a priori, indicating 
that there was a difference in critical 
thinking between the level of treatment 
when controlling for critical thinking pre- 
test scores and MAP scores. An effect size 
of .27 was determined. According to Cohen 
(1977), an effect size greater than .15 is 
considered “large”. 
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Table 4 


Analysis of Covariance (ANCOVA) in Critical Thinking by Instructional Strategy 


Source 

Sum of Squares 

df 

Mean Square 

F 

p - value 

Instructional Strategy 

505.05 

3 

168.35 

10.96 

.01* 

MAP 

139.24 

1 

139.24 

9.06 

.01* 

Pre-test 

134.98 

1 

134.98 

8.79 

.01* 

Error 

1351.94 

105 





Note. Adjusted R Squared = .25 
*p < .05 


The null hypothesis stating that no 
difference existed between groups on critical 
thinking scores was rejected. However, 
treatment group means indicated students in 
the supervised study group scored higher 
than students in the PBL group. Therefore, 
the findings did not support the research 
hypothesis and favored an alternative 
explanation. 

The second research hypothesis stated a 
significant difference exists in content 
knowledge for students taught using the 
problem-based learning instructional 
strategy and students taught using the 
supervised study strategy. ANCOVA was 
used to test the null hypothesis that there 
was no significant difference in content 


knowledge between groups when controlling 
for pre-existing knowledge (quail 
management pre-test) and academic 
aptitude (MAP score) (Table 5). The 
ANCOVA resulted in an F- value (F 2 , 105 = 
14.74) for content knowledge that was 
significant (p = .01) at the alpha .05 level. 
The effect size (.43) was large (Cohen, 
1977). The null hypothesis was rejected 
in favor of the research hypothesis that, 
there was a difference between groups on 
content knowledge scores when 

controlling for pre-test content knowledge 
scores and MAP scores. Students taught 
using supervised study scored higher on 
content knowledge than students taught 
using PBL. 


Table 5 


Analysis of Covariance (ANCOVA) in Content Knowledge by Instructional Strategy 


Source 

Sum of Squares 

df 

Mean Square 

F 

P 

Instructional Strategy 

1315.99 

3 

438.66 

14.74 

.01* 

MAP 

429.37 

1 

429.37 

14.42 

.01* 

Pre-test 

90.11 

1 

90.11 

3.03 

.01* 

Error 

2678.45 

105 





Note. Adjusted R Squared = .3 1 
*p < .05 


Conclusions/Implications/ 

Recommendations 

In the study, 35% of the students were 


female and 65% were male. Data from the 
Missouri Department of Elementary and 
Secondary Education (DESE, 2005) 
indicated approximately 30% of students 
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enrolled in secondary agriculture classes for 
2003-04 were female and approximately 
70% were male. Therefore, it can be 
concluded that the sample approximates the 
gender distribution of secondary agriculture 
students in Missouri. State reports in 2004 
indicated the 40% of students state -wide 
were performing at or above the nearing 
proficiency level on the MAP. Forty-four 
percent of the students in this study were 
classified as nearing proficiency or higher. 
The MAP achievement levels imply students 
in agriculture courses are perfonning at least 
as well as the state-wide population of 
students. 

ANCOVA procedures indicated a 
significant difference between treatment 
groups on critical thinking ability when 
controlling for pre-test critical thinking 
scores and academic aptitude. However, 
post-test means between the treatment 
groups differed only by .2. Therefore, it was 
concluded that there is no practical 
difference between treatments on critical 
thinking ability. These findings contrast 
previous studies (Albanese & Mitchell, 
1993; Hmelo, 1998) that concluded students 
in PBL courses outperformed traditional 
students in problem-solving ability, a 
component of critical thinking. An even 
more direct contrast exists between findings 
from this study and Burbach et al. (2004). 
They concluded that instructional strategy 
resulted in improved critical thinking skills 
as defined by the Watson-Glaser Critical 
Thinking Appraisal®. 

The lack of change between pre-test and 
post-test administrations may be attributed 
to a combination of factors. This study 
focused on a single unit of instruction 
implemented over a two-week time period. 
Most likely, discrepancies between this 
study and studies that found instructional 
strategy to effectively increase critical 
thinking ability (Burbach et al., 2004; Elliot 
et al., 2001; Lundy et al., 2002; Mabie & 
Baker, 1996;) can be explained by the 
relative short treatment length of this 
study. Treatment lengths in those studies 
ranged from 10 to 16 weeks. It is likely that 
the relatively short duration of the 
treatment was not sufficient to detect 
any differences likely caused by that 


treatment. It is possible that extending the 
length of treatment may yield different 
results. 

From the findings related to content 
knowledge, it can be concluded that students 
in supervised study classes tended to score 
higher on content knowledge assessments 
than students in PBL classes. Students in 
the supervised study group scored an 
average of 5 points higher than students in 
the PBL group on content knowledge and an 
average of almost 9 points higher than their 
pretest scores. Students in PBL classes 
showed an improvement of just over 4 
points of their pre-test scores. These 
findings are consistent with other 
studies that found PBL students did not 
perform as well on knowledge exams 
(Albanese & Mitchell, 1993; Vernon & 
Blake, 1993). 

Dods (1997) concluded that more 
traditional approaches to instruction 
promoted content coverage. This may 
provide some explanation of the success of 
those students on assessment items 
associated with recall and identification of 
content material. While PBL students may 
have a deeper understanding of the material, 
that understanding is not represented at a 
content knowledge level. 

Treatment lengths were equal between 
groups for this study, yet a significant 
difference in content knowledge was found. 
The history of problem-solving in 
agricultural education is well established, 
but the use of PBL is a relatively new 
approach. Students in the PBL classes may 
have experienced some discomfort in 
adjusting to a new strategy for 
instruction. Ryan and Millspaugh (2004), in 
their model of PBL, describe step 1 of 
instruction as a description of why PBL is 
used. It can be argued some time on task 
was lost due to learning an unfamiliar 
process. Extended exposure to the 
treatment may offset this learning 

process and better detect effects on student 
outcomes. This study should be replicated 
with an increased treatment period. Further 
investigation may provide insight into the 
effects that instructional strategies can have 
on student outcomes over a longer 
duration. 
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